this should be before the solve, since you can use the initialization solution from here in the remakes of 397-405 in order to set new u0 and p and thus skip running the initialization a second time.

How can I indicate to solve to avoid running initialization?

initializealg = NoInit(). Should probably just do CheckInit() for safety but either is fine.

Yeah I don't know about that comment. I think it's just old. (a) \ always allocates because it uses lu instead of lu!, so it's re-allocating the while matrix which is larger than any LinearSolve allocation, and (b) we have since 2023 setup tests on StaticArrays, so the immutable path is non-allocating. I don't think (b) was true when this was written.

So glad we can remove this branch altogether.

This gradient isn't used? I think this would go into the backpass and if I'm thinking clearly, the resulting return is dp .* igs?

Not yet. These gradients are currently against the parameters of the initialization problem, not the system exactly. And the mapping between the two is ill defined, so we cannot simply accum

I spoke with @AayushSabharwal about a way to map, it seems initialization_data.intializeprobmap might have some support to return the correctly shaped vector, but there are cases where we cannot know the ordering of dp either.

There's another subtlety. I am not sure we haven't missed some part of the cfg by manually handling accumulation of gradients. Or any transforms we might need to calculate gradients for. The regular AD graph building typically took care of these details for us, but in this case we would need to worry about incorrect gradients manually

Oh yes, you need to use the initializeprobmap https://github.com/SciML/SciMLBase.jl/blob/master/src/initialization.jl#L268 to map it back to the shape of the initial parameters.

but there are cases where we cannot know the ordering of dp either.

p and dp just need the same ordering, so initializeprobmap should do the trick.

There's another subtlety. I am not sure we haven't missed some part of the cfg by manually handling accumulation of gradients. Or any transforms we might need to calculate gradients for. The regular AD graph building typically took care of these details for us, but in this case we would need to worry about incorrect gradients manually

This is the only change to (u0,p) before solving, so this would account for it, given initializeprobmap is just an index map so an identity function.

Addressed this occurance in 95ebbf3 to check if this is correct. Will need to work around the global call

shouldn't default to ZygoteVJP. Should use the autojacvec of the ODE

Addressed in 9a8a845

same down here?

These don't make sense.

Yes, these should probably inherit from kwargs or be set up to some default. Note that we must specify a tol for this dispatch.

Addressed in 984c2ce

Also needs to check that initializealg is not set, is the default, or is using OverrideInit. Should test this is not triggered with say manual BrownBasicInit

Got it. My understanding was that OverrideInit was what we strictly needed. We can check for BrownBasicInit/ defaults here.

There doesn't seem to be a method which can take a BrownFullBasicInit(). I get a MethodError:

ERROR: MethodError: no method matching get_initial_values(::ODEProblem{…}, ::ODEProblem{…}, ::ODEFunction{…}, ::BrownFullBasicInit{…}, ::Val{…}; sensealg::SteadyStateAdjoint{…}, nlsolve_alg::Nothing)

Closest candidates are:
  get_initial_values(::Any, ::Any, ::Any, ::NoInit, ::Any; kwargs...)
   @ SciMLBase ~/Downloads/arpa/jsmo/t2/SciMLBase.jl/src/initialization.jl:282
  get_initial_values(::Any, ::Any, ::Any, ::SciMLBase.OverrideInit, ::Union{Val{true}, Val{false}}; nlsolve_alg, abstol, reltol, kwargs...)
   @ SciMLBase ~/Downloads/arpa/jsmo/t2/SciMLBase.jl/src/initialization.jl:224
  get_initial_values(::SciMLBase.AbstractDEProblem, ::SciMLBase.DEIntegrator, ::Any, ::CheckInit, ::Union{Val{true}, Val{false}}; abstol, kwargs...)
   @ SciMLBase ~/Downloads/arpa/jsmo/t2/SciMLBase.jl/src/initialization.jl:161
  ...

Only CheckInit, NoInit, and OverrideInit have dispatches.

Having chatted with @AayushSabharwal on this, it seems like BrownBasic and ShampineCollocation do not yet have a path through get_initial_values and that would need to be fixed in OrdianryDiffEq. Further, as SciMLSensitivity does not depend on OrdinaryDiffEq, it cannot check for whether there is a default initialisation.since those are defined there. Depending on it also seems like a pretty big hammer for a dep.

What would be the best course of action here? Seems like supporting BrownBasicInit is a dispatch that will automatically be utilised when it is moved into SciMLBase.

Yes I mean BrownBasicInit should not be taking this path. But that's a problem because then they will be disabled in the next stage below, and that needs to be accounted for. This dispatch is already built and setup for BrownBasicInit and there are tests on that.

It should only noinit if the previous case was ran. Won't this right now break the brownbasic tests?

If there was no intialization data, it won't have ran the initialization problem at all.

If I can genetically ignore handling initializealg and pass it directly to get_initial_values, that would be good. Then I can also pass NoInit here genetically.

No that is not correct. If there was no initialization data then it will use the built in initialization, defaulting to BrownBasicInit. It's impossible for a DAE solver to generally work without running initialization of some form, the MTK one is just a new specialized one but there has always been a numerical one in the solver. And if it hits that case, this code will now disable that.

https://github.com/SciML/SciMLSensitivity.jl/blob/master/test/adjoint.jl#L952-L978 this code will hit that. I think it's not failing because it's not so pronounced here. You might want to change that test to https://github.com/SciML/SciMLSensitivity.jl/blob/master/test/adjoint.jl#L975C5-L975C69 prob_singular_mm = ODEProblem(f, [1.0, 0.0, 1.0], (0.0, 100), p) and it would pass before and fail now.

You're right of course for the DAEs, but since BrownBasicInit is defined in OrdinaryDiffEq, and this package does not depend on it, I need a way for us to be able to dispatch to it. So if I understand the comment from earlier, we need a check for the default initialization, and add a branch that solves for that prob, and collect all the outputs.

Both BrownBasicInit and OrdinaryDiffEqCore.DefaultInit require us to depend on a whole package for the default dispatch. Can it be exposed as a dispatch of get_initial_values instead?

Oh 😅 . That case was too simple, MTK turns it into an ODE. Let's make it a DAE.

@parameters σ ρ β A[1:3]
@variables x(t) y(t) z(t) w(t) w2(t)
eqs = [D(D(x)) ~ σ * (y - x),
    D(y) ~ x * (ρ - z) - y,
    D(z) ~ x * y - β * z,
    w ~ x + y + z + 2 * β
    0 ~ x^2 + y^2 - w2^2
]
@mtkbuild sys = ODESystem(eqs, t)

That should make it so that it eliminates the w term, but doesn't eliminate the w2 term. The DAE check is on the w2 term, the observed handling check is on the w term.

That will need to change the integrator to Rodas5P, Tsit5 will not be compatible with this form.

For SDEs, we will just need to make it compatible with BrownBasicInit.

Ah, I was so confused why it worked out, I see the InitialFailure now.

Okay good. Yeah because MTK is too smart and makes lots of simple examples not DAEs 😅. But now you got the DAE, and if not running the built in init then you get the error I was expecting. The fix is that it needs to run brownbasic before solving for the same reason reverse needs to. Good we worked out a test for this

When the new reverse ode is built it needs to drop the initial eqs but still keep the dae constraints. It can brownbasic?

Is there a way to drop the initial eqs after its solved? The assumption was since we run with NoInit, no initialization is run post the first call to get_initial_values and we accumulate those gradients independently of the adaptive solve.

But the reverse pass needs to run with some form of initialization or the starting algebraic conditions may not be satisfied. Don't run this one with NoInit(), that would be prone to hiding issue. For this one, at most CheckInit(), but I'm saying that BrownBasicInit() is likely the one justified here since the 0 initial condition is only true on the differential variables, while the algebraic variable initial conditions will be unknown, but the Newton solve will have zero derivative because all of the inputs are just Newton guesses, so BrownBasic will work out for the reverse. We should probably hardcode that since it's always the solution there.

Ok, that will require us to add an OrdinaryDiffEqCore dep in this package. I will add that.

Is the 0 derivative also applicable to parameters? Or only the unknowns?

Its applicable to all Newton guess values. There is no parameter init going on to reverse so it's only for algebraic conditions so it's only Newton guesses.

We should test OverrideInit and DefaultInit